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Optimization  algorithms  on  random  graphs* 

D*  G#  Kelly*  and  D«  J.  A.  Welsh 

I*  Introduction* 

Let  G  be  a  graph  on  vertices  0,  1#  n-1.  We  consider  two  'algorithms' 

which  start  at  an  arbitrary  vertex#  move  at  each  step  to  a  lower-numbered 
neighboring  vertex#  and  stop  when  they  reach  a  sink#  i#e#  a  vertex  with  no 
lower-numbered  neighbor# 

The  oreedy  algorithm  moves  from  a  given  vertex  to  the  lowest-numbered 
neighbor} 

The  random  algorithm  moves  from  a  given  vertex  to  a  vertex  chosen  at 
random  from  the  lower-numbered  neighbors# 

We  are  interested  in  the  behavior  of  these  two  algorithms  on  random 

* 

graphs.  We  use  the  model  of  random  graphs  in  which  each  of  the  ( £  )  edges  is 
present  or  absent  wi*h  probabilities  p  and  q=l-p  respectively .  independently  of 
the  other  edges.  Such  a  random  graph  will  be  called  a  Bern(n,p)  graph.  We 
will  consider  the  following  random  variables. 

S^,  the  number  of  sinks  in  a  Bern(n»p)  graph. 

G^»  the  number  of  steps  from  vertex  k  to  a  sink  using  the  greedy  algorithm 
on  a  Bern(n.p)  graph! 

R^.  the  number  of  steps  from  vertex  k  to  a  sink  using  the  random  algorithm 
on  a  Bern(n,p)  graph. 

These  considerations  are  motivated  by  probabilistic  analysis  of  linear 
programming,  which  attempts  to  reconcile  the  widely-observed  efficiency  of  the 
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I  Abstract 

v 

He  consider  random  graphs  on  vertices  0,  1#  2,  n  in  which  each  edge, 
independent  of  the  others,  is  present  with  probability  p  and  absent  with 
probability  q=l-p.  On  such  a  graph  we  consider  two  different  random  walks 
starting  with  vertex  n,  moving  at  each  step  from  a  vertex  to  a  lower-numbered 
neighbor,  and  stopping  when  they  reach  either  vertex  0  or  a  sink,  a  vertex  with 
no  lower-numbered  neighbor*  These  walks  are  simplified  attempts  to  reproduce 
probabilistically  the  behavior  of  the  simplex  algorithm  on  linear  programs. 

We  derive  some  simple  asymptotic  results  on  the  distribution  of  the  number 
of  sinks  in  a  random  graph,  and  also  on  the  distributions  of  the  numbers  of 
steps  needed  by  each  of  the  two  random  walks. 


simplex  algorithm  with  the  existence  of  problems  on  which  it  is  exponentially 
inefficient. 

IF  a  linear  program  in  d  variables  with  m  constraints  is  generated  at 
random  from  one  of  a  wide  class  of  distributions  CDKT],  its  feasible  region  is 
with  probability  one  a  d-dimensional  polyhedron  that  is  simple  teach  vertex  has 
exactly  d  neighbors),  in  which  no  two  vertices  have  the  same  objective  value. 
Thus  it  is  possible  to  number  the  vertices  0,  1,  ♦*.,  n-1  so  that  0  is  the 
optimal  vertex  and  so  that  from  any  vertex,  any  simplex  pivot  moves  to  a 
lower-numbered  neighboring  vertex. 

In  another  approach  to  probabilistic  analysis  £K],  one  considers  a  fixed 
linear  program  and  regards  the  simplex  algorithm  as  a  random  walk  on  the  graph 
of  its  feasible  region  which  moves  always  in  the  direction  of  improvement  in 
objective  value.  This  is  obviously  the  motivation  for  the  random  algorithm 
described  above. 

Our  graph-algorithm  models  fail  to  match  the  above  situations  in  two 
respects.  First,  the  random  graphs  generated  are  not  regular  as  are  the  graphs 
of  simple  polyhedra.  This  is  a  relatively  minor  defect,  however:  by  proper 
choice  of  p  and  n  one  can  adjust  the  average  degree  so  that  the  actual  vertex 
degrees  will  be  close  to  dj  and  one  feels  that  it  is  not  the  regularity,  but 
rather  the  fact  that  the  vertex  degrees  are  bounded  below,  that  contributes  to 
the  efficiency  of  pivoting  procedures.  Second,  and  more  important,  the  random 
graphs  generated  will  in  general  have  many  sinks,  while  the  polyhedron  graphs 
have  only  one.  Nevertheless  we  feel  that  an  investigation  of  the  behavior  of 
such  algorithms  on  graphs  may  shed  some  light,  at  least  on  the  combinatorial 
reasons  of  the  simplex  algorithm's  efficiency  <as  opposed  to  reasons  related  to 
convexity). 
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Another  difference  between  random  graphs  and  random  linear  programs  is 

that  the  former  have  a  fixed  number  of  vertices  (and  random  vertex  degree) 

while  the  latter  have  fixed  vertex  degree  but  a  random  number  of  vertices*  The 

distribution  of  the  number  n  of  vertices  is  not  known*  although  one  has  CKleel 

(m-d)(d-l)  +  2  <  n  <  (m~f)  +  (m~^) 

m-a  m-d 

(where  a  and  b  are*  respectively,  the  greatest  integers  in  <d+l)/2  and 
(d+2)/2),  and  for  a  certain  class  of  distributions  on  polyhedra  with  m  facets 
in  d  dimensions  CKT1 

j  /O 

E(n)  "•  (cd)  m  as  m  cd  for  large  d 
where  c  is  a  constant  independent  of  d  and  m. 

In  a  Bern(n,p)  graph  the  average  vertex  degree  is  approximately  np*  and  so 
we  might  be  interested  in  the  case  of  large  n  and  P  =  Pn  satisfying 
np  •'  d  and  n  *  d  * 


i.e. 


n  ~  (np) 


np/2 

♦ 


II.  Sinks  in  random  acyclic  directed  graphs. 

A.  Let  G  be  a  Bern(n.p)  graph  with  vertices  0,  1 ,  ....  n-1,  and  regard 

any  edge  as  directed  from  the  higher-  to  the  lower-numbered  vertex.  A  vertex 

of  outdegree  zero  is  a  sink}  we  let  S  *  „  be  the  number  of  sinks  in  G.  (S 

n»p 

is  at  least  1  because  vertex  0  must  be  a  sink.)  Define  random  variables  X. 


(j=0,...,n-l)  by 


1  if  j  is  a  sink. 


0  if  not. 

So  S  =  Xjt  the  Xj  are  independent,  and 

EX.  =  Pr(X=l)  =  q>. 

1  1 


It  follows  immediately  that  the  characteristic  function  and  the  probability 
generating  function  of  S  are,  respectively, 

gjg(t)  =  E(eltS)  =  TT^  U-U-e^lq*). 


We  also  have 


ns(u)  =  E(uS)  =  TT"~q  (t-(l-u)q*). 


ES  =  l+q+q2+...+qn  *  =  (l-qn)/p 


and,  since  VX^  =  q^(l-q^)  and  the  X^  are  independent, 
VS  =  q(l-qn-1)(l-qnHl-q^r1 


VS/(ES)2  =  pq(l-qn  1  )<l+q>~1(l-qn,-1. 

Notice  also  that  S  is  asymptotically  normal  for  large  n. 

B.  Now  we  consider  p  =  pn,  q  =  q^  =  1-p^  as  functions  of  n  and  look  at 
the  behavior  of  S  =  Sn  as  n  increases  for  various  orders  of  magnitude  of  pn« 

1*  If  Pn  tends  to  a  constant  p  (0<p<l)»  then  ES^  approaches  1/p  and  VSn 
approaches  q/p(l+q)  (where  q*i-p),  and  the  distribution  of  Sn  tends  to  the 
normal  distribution  with  these  parameters.  (In  general,  if  pn  is  bounded  away 
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from  Of  then  ESn  -  l/pn  and  VSn  -  c^/p^l+q^)  both  approach  zero,) 

2.  If  p„  *4  0  as  n  cd»  then  VS  /(ES  r  -4  0  also,  and  it  follows  that 
n  n  n 

S  /ES  -3*  1  in  probability,  Ke  consider  the  asymptotic  behavior  of  ES  for  a 
n  n  n 

few  growth  rates  of  p^* 
a*  If  p  *  (log  n)/n#  then 

ESn  *  (n/log  nXl  -  (l-(log  n)/n)n) 

Mn/lognHl-n  S 
'  n/log  n. 

b.  If  np_  -4  at  <0<a<ai),  then 

n 

ES„  '  ?'\,  - 

'vPn  *(l-e  a) 

^(l-e  a)n/ac, 

(k+l)/k 

c.  If  np  -4  0  but  n  -4  a  (0<a<co),  then  since 

n 

ESn  =  n-(")p  +  <5)p2-+... 

and  since  the  partial  sums  in  this  expression  alternately  over-  and 
underestimate  ES^*  eventually  ES^  is  between 
n  -  (^Jp  +  “  ***  ±  <  ^  *  P 
and 


,  n  v  « n ,  k>"~  1  /  n  , 

» -<2>p  *-»iik)p  ‘ikM>p 

—  k 

and  the  last  term  on  the  right  approaches  +a  asn  4  cd. 


k. 


* 

l 
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III,  Number  of  steps  needed  by  greedy  algorithm. 

Again  fix  n,  p,  and  k  (k  =  0,l,.».,n-l)»  Gk  is  the  number  of  steps  from  k 
to  a  sink  using  the  greedy  algorithm  on  a  Bern(n,p)  graph. 

k 

Vertex  k  is  a  sink,  and  hence  »  0,  with  probability  q  ,'  otherwise  the 
first  step  is  to  vertex  j  with  probability  pq*  (j  =  0»l,...,k-l).  Consequently 
the  characteristic  function  of  Gk  is 


...  itG.  ,  k  .  — k-1  j  it(l+G.) . 

cp^(t)  =  E(e  k)  =  q  +  >  ._ j  pqJ  E(e  )  ), 


...  k.  it^-k-1 
”k,tl  * q  *e 

Replacing  k  by  k+1  in  (2)  and  subtracting  (2)  from  the  result  yields 

...  #1,  k+1  kx  it  k  ... 

<Pk+1<t)  -  «Pk<t>  =  q  -q  +e  pq  q>k<t), 

whence 

qj^+1(t)  s  (l+pe^qS  cp^tt)  -  pqk  (k>0), 

cp0(t)=  1. 

Similarly,  for  the  probability  generating  function, 

TTk+1(u)  =  (l+pqku)nk<u)  -  pqk. 

A*  To  get  an  expression,  for  write  p^(t)  =  e^qD^(t)  -  1*  Then  one 


Pk+i(t)  =  Pk<t)a  +  pqkelt)* 

Since  pJt)  =  e**-l  we  have 


pk(t>  =  (e^-Dir^  (1  +  pqV*) 


and  hence 


(pk<t)  =  e  **  +  (1  -  e  **)TTk_*  (1  +  pqV*L 
B.  Let  pk  =  EGk  =  ).  Differentiating  (3)  and  setting  u  =  1  gives 

Pk+1  =  Pqk+(1+Pqk)Pk* 

Writing  ak  =  pk  +  1  we  get 


Vi  ’ 


I 
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so  that 


and 


ak=ir1=ia+p(i1) 


-Vrld+pqf)-!. 


Pk"  ”  j=0 

Notice  that  regardless  of  the  values  of  n  and  p»  because  of  the  arithmetic  mean 


-  geometric  mean  inequality  we  have 

1  +  pk  <  (1  +  p(l+q+q2+...+qk-1)/k)k 


=  (1  +  U-qS/k^ 

c  n  +  i  n 


IV*  Number  of  steps  needed  by  random  algorithm* 

Let  be  the  number  of  steps  from  k  to  a  sink  using  the  random  algorithm 

on  a  Bern(n»p)  graph*  With  probability  q  vertex  k  is  a  sink  and  =  0*  With 
k 

probability  1-q  vertex  k  is  not  a  sink}  in  this  case  by  symmetry  all  vertices 
of  lower  index  than  k  have  the  same  chance  of  being  visited  next.  Accordingly 

the  characteristic  function  of  R^  is 

i  ....  tv  itR  .  k.  , -iM  k.— k-1  it(R.+l). 
di^tt)  =  E(e  k.)  =  q  +  k  (1-q  E(e  3  )» 

i.e. 

,  ...  k  ,  ,-l,4  _k.  it=r-k-l  .  ... 

<|^(t)  =  q  +  k  (1-q  )e  I>  ^  ik(t). 

Similarly,  the  probability  generating  function  is 

X.  (u)  =  qk  +  k~*(l-qk)u>k_i  X.(u). 

Hence,  if  p^  =  ER^  =  xyi),  then  p^  =  0  and 

A*  To  get  a  quick  upper  bound  on  p^  (a  better  bound  will  be  obtained  in 
B*  below)  we  note  that  if  is  defined  by  the  recurrence 
ock  =  1  +  k_1  >k“*  cij  (with  a0  =  0), 
then  it  is  easy  to  show  that  p^  <  and  also  that 
(k+1)ak+i~k“k=1+V 

whence 

ak"vi*  */k' 

Thus  we  have 

>jl,  1/j  %  log  k. 

B.  For  more  precise  estimates  of  p^  we  write 


so  that 

'k'£u<1'k'1^'i>- 

Putting  k+1  in  place  of  k  in  (4),  subtracting  (4)  from  the  result,  and 


page  9 


simplifying »  we  find 

pk+l  =  rk+lPk  +  Sk+l' 

where 

and 

-1 

5.  =  ]  C# 

3  3 

It  follows  easily  that 

Pk =  5k +  rkVi +  rkrk-i\-2 +  ••• +  rkW*r2sr  (5) 

Moreover  5 

sk=  k  ^l-qS, 

rksk-l  =  <k"1)~1(1_tlkMl-k~1qk~1>» 

and  in  general 

rk,»r.+1s.  =  j'1(l-qSa-k“1qk“1>«-(k-ir1qk_2)...(l-<j+irV)‘  (6) 

It  follows  easily  that 

1/j  *  (l-qk>  log  k* 

A  number  of  different  lower  bounds  on  pk  can  be  obtained  from  (5)  and  (6). 

Among  them  are! 

n  .  k 
Pk>  1-q  . 

^id-qSd-q^^lj  1/j, 

>  a-qk)(2  +  q-1logU-q)>  >k  (  1/j  . 


and 
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V*  Open  problems 

(There  is  no  guarantee  that  any  of  the  problems  posed  here  are  difficult, 
or  that  their  solutions  will  constitute  major  progress*  There  is  also  no 
guarantee  that  any  of  them  are  well-posed  or  tractable*) 

A*  For  the  greedy  algorithm,  find  a  closed-form  expression  for  the 
characteristic  function  cp^(t)  of  the  number  of  steps  from  k  to  a  sink*  Also 
find  the  variance  of  G^,  and  get  asymptotic  values  for  the  mean  and  variance* 
Show  that  the  expected  value  is  an  increasing  function  of  k,  and  also  an 
increasing  function  of  p* 

B*  For  the  random  algorithm,  attempt  to  find  a  closed-form  expression  for 
the  characteristic  function  ij^(t)  of  the  number 
Find  a  closed-form  expression  for  the  expected  value  p^>  and  for  the  variance 
as  well*  Find  asymptotic  expressions  or  bounds  on  these  quantities*  Show  that 
the  expected  value  p^  is  increasing  as  a  function  of  k  and  of  p,  Find  an 
asymptotic  lower  bound  on  p^  (log  k  would  be  nice)* 

C*  Look  into  controlling  the  number  of  sinks  in  the  random  graphs  under 
consideration,  either  by  adjusting  p  as  a  function  of  n  (probably  this  can  be 
done  only  with  unreasonable  values  of  p)  or  by  changing  the  model  somehow* 
Look  into  threshold-function  results  for  various  behaviors*  for  example,  is 
there  a  threshold  growth  rate  on  p  =  pn  for  boundedness  of  the  sequence  ? 


of  steps  from  k  to  a  sink. 
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